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Abstract. 



Consider a particle that moves on a connected, undirected graph G with n vertices. At 
each step the particle goes from the current vertex to one of its neighbors, chosen uniformly 
at random. The cover time is the first time when the particle has visited all the vertices 
in the graph starting from a given vertex. 

In this paper, we present upper and lower bounds that relate the expected cover 
time for a graph to the eigenvalues of the Markov chain that describes the random walk 
above. An interesting consequence is that regular expander graphs have expected cover 
time 0(n log n). 
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1. Introduction. 



Consider a particle moving on an undirected graph G = (V } E) from vertex to vertex 
according to the following rule: the probability of a transition from vertex z, of degree di, 
to vertex j is 1/di if G E, and 0 otherwise. This stochastic process is a Markov chain; 
it is called a random walk on the graph G. In this paper we derive upper and lower bounds 
on the expected cover time, the time taken by a random walk on G to visit all vertices in 
V . (See [11] for a general reference on Markov chains.) 

Formally, let {X t } be any discrete time Markov chain with state space S. (For the 
random walk S = V above.) The hitting time Hij for i,j ' G S is defined by 

Hij = inf {< : X t = j | X 0 = i} 

and the cover time (or covering time) d, for i G S is defined as 

C i = max Hi j . 

In other words, the hitting time Hij is the first time that state j is reached starting from 
state z, and the cover time d is the first time every state in S is visited at least once 
starting from state i. 

Computer scientists originally became interested in analyzing the expected cover 
times for graphs in an attempt to obtain bounds on the space complexity of undirected 
st- connectivity [6] (Given an undirected graph G and two specified vertices s and t in G } 
determine if there is a path connecting s and t). There are several other topics in computer 
science and graph theory that motivate the investigation of cover times: 

• The study of the relations between combinatorial properties of graphs (such as cover 
times, expansion etc.) and algebraic properties of graphs (the eigenstructure of the 
associated adjacency matrix). Some recent papers in this long line of research are [1], 
[7], and [9]. 

• The exploration of the special characteristics of expander graphs such as short covering 
times, rapidly-mixing properties, the existence of short universal sequences for them, 
etc. 

• The simulation of token rings on arbitrary networks using random redirection of to- 
kens: There is a plethora of protocols and algorithms designed for use on token ring 
networks. It is desirable to be able to simulate these protocols on arbitrary networks 
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and most notably on some of the extremely high-speed networks with limited buffer 
capacity that are currently being built. An obvious approach for doing such a simu- 
lation is simply to route the token along a spanning tree of the network. A problem 
with this idea is that it is not fault-tolerant; when some nodes or links become faulty 
or flooded, an extensive reconfiguration of the spanning tree might be required while 
the old ring is not operative. These operations entail rather complicated and slow 
mechanisms. An alternative approach, randomly routing tokens, is efficient (if the 
expected cover times are small), extremely fault-tolerant, and suffers little loss of 
efficiency when several "colored" tokens are routed simultaneously. 
There is a large body of previous work on cover times and related problems. First, 
there is the classical solution to the coupon collector's problem, which shows that the 
expected time to collect n coupons is ©(nlogn). This is essentially equivalent to showing 
that the expected cover time for a random walk on the complete graph is 0(n log n). There 
are many other specific graphs for which the expected cover time has been computed. These 
include paths, cycles, trees [10], bar-bell graphs 1 [14], and (/-dimensional cubes [4]. 

For arbitrary connected graphs, Aleliunas et al. [6] showed a general upper bound 
E(Ci) = |V"|), starting from any vertex z, where \E\ is the number of edges and |V| 

is the number of vertices. 

A superficial examination of these problems might lead one to conjecture that adding 
more edges to the graph would reduce the expected cover time. This is false. For example, 
the complete graph, K n , on n vertices, has expected cover time O(nlogn); the bar-bell 
graph B n , on n vertices has expected cover time fi(n 3 ); and the line graph L n , consisting of 
a path of length n — 1, has expected cover time 0(n 2 ). On the other hand K n D B n D L n . 

In this paper, we present relations between the expected cover time for graphs and 
the rate of convergence of the corresponding Markov chain to its stationary distribution, 
which in turn is determined by the second largest eigenvalue of its transition probability 
matrix. A preliminary version of part of this work has appeared in [8]. Most theorems 
refer to reversible Markov chains, that is, Markov chains for which TTiPij = TTjPji, where 
7r is a stationary distribution, and P is the transition probability matrix. A random walk 
on a graph is a particular case of a reversible Markov chain: a stationary distribution is 
vr, = di/(2\E\); if e E then ir.P^ = vr^ = 1/(2 \E\); if £ E then ir.P^ = 

1 The bar-bell graph on n vertices consists of two cliques each of size n/3, connected by 
a path of length n/3. 



2 



TTjPji = 0. 

Our main results are the following: 

Theorem 5. Let M be an irreducible reversible Markov chain on n states with transition 
probability matrix P. Let 1 = Ai > A2 > • • • > \ n be the eigenvalues of P. Let tt = 
(7Ti,7T2, . . . ,7r n ) be the eigenvector corresponding to the eigenvalue 1, normalized so that 
TTi = 1. (If M is aperiodic then tt is the stationary distribution of M.) We rename the 
states such that tt\ < 7T2 < • • • < 7r n . Let e > 0 be a constant. Then the cover time C 
starting from any state satisfies 

E(C)<^-((2 + e)nlnn-nlnir 1 + ]T ( ]T 7ri) _1 Vl + 0 (l)). 

2 ^ l<i<n 1<J<1 ' 

I 

This theorem has several interesting consequences: 

1. Any symmetric, irreducible Markov chain has expected cover time 0(n log n/(l — A2)) . 
(A Markov chain is symmetric if its transition probability matrix is symmetric.) 

2. Any (/-regular expander graph has expected cover time O(nlogn). 

3. Any connected graph has expected cover time 0(n 2 logn/(l — A2)). 
We also obtain two lower bound results: 

Theorem 13. Let M be an irreducible and aperiodic, reversible Markov chain on n 
states with transition probability matrix P. Assume that P is doubly-stochastic and 
has eigenvalues 1 = Ai > A2 > • • • > A n . Let A max = max2< 8 <« | Ai |. Suppose that 
Amax < 1 — n e_1 for e > 0. Then the cover time C , starting from any state, satisfies 
E(C) = O(nlogn). | 

Corollary 16. Let M be an irreducible and aperiodic, reversible Markov chain on n 
states, with transition probability matrix P. Let 1 = Ai > • • • > \ n be the eigenvalues 
of P. If A2 > 1 — l/(nlnn) then the expected cover time starting from the stationary 
distribution of M is O(nlogn). | 

The two results above can be generalized to periodic Markov chains, but we omit the 
proofs. For random walks on graphs, "the chain is aperiodic" is equivalent to "the graph 
is not bipartite," and "the transition probability matrix is doubly-stochastic" is equivalent 
to "the graph is regular." 



3 



Aldous [5] has recently showed a general lower bound for random walks on graphs, 
E(C) = 0(n log n) starting from the stationary distribution, but not necessarily from every 
state. 

2. The upper bound. 

In this section, we present an upper bound on the expected number of steps needed to visit 
all states in a reversible Markov chain with state space V . We decompose this process into 
a sum of expected first passage times, each from a set of states S that the random walk 
has already visited to a state in V — S . These expected first passage times are bounded 
by a function of the eigenvalues of the corresponding transition submatrix. The critical 
new result needed to exploit this bound is Lemma 1(b) below, which relates the largest 
eigenvalue of this submatrix to the second largest eigenvalue of the complete matrix. 

Lemma 1. Let M be an irreducible and aperiodic, reversible Markov chain on n states 
with transition probability matrix P. Let S be a subset of the states of size s. Let R be 
the submatrix of P corresponding to the set of indices S X S. Then 

(a) All the eigenvalues of P and of R are real. 

(b) Let the eigenvalues of P be 1 = Ai > A2 > • • • > A n; let A max = max2< 8 <« |Ai|, and 
let the eigenvalues of R be \i\ > ^2 > • • • > Hs- Then 

1 - Hi > (1 - A max ) - ^ TTz) 

i£S 

where (7Ti, . . . , Tr n ) is the stationary distribution of M. 

(c) 

Hi ^ ^max- 

Proof: Part (a) is standard but we present its proof for completeness. Let D be the 
diagonal matrix given by Dn = 1/tvi. Then P = DA where A is given by Aij = TTiPij 
and is symmetric because of reversibility. It is easy to show that P has exactly the same 
eigenvalues as the symmetric matrix B = D 1 ! 2 AD 1 ! 2 . Indeed, let x be a left eigenvector 
of P corresponding to an eigenvalue A. Then xDA = \x implies (xD 1 / 2 )D 1 / 2 AD 1 / 2 = 
X^xD 1 ! 2 ). Since B is symmetric, it has real eigenvalues and hence P does. Similarly, R 
has the same eigenvalues as the symmetric submatrix Q C B induced by the set of indices 
S x S. 



4 



We now prove part (b): A left eigenvector of B corresponding to the stationary 
distribution of P is a = (-^/vrT, \/t^2, • • • , \/^n)- Let p be a positive left eigenvector of Q for 
eigenvalue p\ (such a p exists by a weak analogue of the Perron- Frobenius theorem), that 
is pQ = pip. Pad p to an n element vector by adding zeros for all indices not in S. Call 
the resulting vector p. We normalize p such that 

(p,a) = (a,a) = l (1) 

(This is always possible because p and a are positive.) 
We can write 

pB = pip + w (2) 
where w = (ii>i, . . . , w n ) and wi = 0 for all i £ S. The vector w satisfies 

(w,a) = 1 - p 1 (3) 

because 1 = (/?, a) = (/?, aB) = (pB } a) = pi(p } (t) + (w } a). 

By the variational principle, A2 = max x j_ CT (xB } x)/(x,x). Define z = p — a. Since z is 
orthogonal to cr, we obtain that A2 > (zB , z)/(z } z) . Expanding this inequality via (2) and 
(3), we obtain that 

((p - a)B,p - a) (pxp + w - a,p - a) 



A 2 > 



H 



ence, 



(p-a,p-a) (p-a,p-a) 
pi(p, p) - 1 - pi - (1 - pi) + 1 = Pi(p,p) ~ 1 
{p,p)~ l (P,P)~1 

p,!(p,p)-l (p,p)(l-/ii) 



1 - Ao < 1 



or 



{pip)~ l (Pip)- 1 
1 



(l-^i)>(l-A 2 ) 1 



(p,p). 

We now need a lower bound on (/?, p). From(l), ~^2 ieS Piy^ = 1, so by Cauchy's inequality, 



TT t . 



Hence, (p,p) > so finally, 

> (1- A 2 )(l- J^TTi) > (1- A max )(l- J^TTi 
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Part (c) is a consequence of the interlacing of eigenvalues of submatrices. (Once again, 
this claim is standard.) To prove the claim, we use another form of the variational formula, 
which states that if \2(M) is the second largest eigenvalue of the symmetric matrix Q } 
and is any linear space of dimension 1, then 

^2(Q) = mm max ^ ' .) . 

Li 1 ) x-LLi 1 ) (x, X) 

Let a be the principal eigenvector of the symmetric matrix B and let the vector a have 
components oi = oi if i £ S, and U{ = 0 if i £" S. Also, let a be the s element vector 
corresponding to the components of a with indices in S. Finally, let x denote an s element 
vector and y denote an n element vector. We have 

(Qx,x) (Qx } x) 
H2 = nim max — — < max — — 

L(!) x±L(!) (X,X) x±cr (X,X) 

(By,y) (By,y) 
max — — < max — — = \i < A r 

(y,y) y-t-o- (y,y) 

I 

The next lemma is standard, but presented for completeness. 

Lemma 2. Let M he an irreducible and aperiodic, reversible Markov chain on n states 
with transition probability matrix P, and stationary distribution (7Ti, 7T2, • • • , 7r n ). Let 1 = 
Ai > A2 > • • • > \ n be the eigenvalues of P. Let S be a subset of the states of size s. Let 
R be the submatrix of P corresponding to the set of indices S X S. Let the eigenvalues of 
R be /ii > /j.2 > • • • > n s and let p\ be a positive eigenvector of R corresponding to 
Then 



y -Lit 



p (t) ^ pijPik __ t ,n( 



where /i max = max 2 <i< s \ni\, and 




Pm 



p jk = ^k + 0[ w — A max 



*3 



where A max = max 2 <,< s |A 4 |. 

Proof: Without loss of generality assume S = Let D be the s by s diagonal 

matrix given by dn = 1/tvi. As in Lemma 1, we associate with R the symmetric matrix Q = 
D -1 / 2 RD 1 / 2 , which has the same eigenvalues as R. Let f3\ , . . . , f3 s be a set of orthonormal 
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eigenvectors of Q with corresponding eigenvalues We can always take f3\ 

p 1 D 1 / 2 /\\p 1 D 1 / 2 \\. 

Applying the spectral decomposition, 

Q ( jk = PijPik»l + PhPlkrf < PijPiklA + I^'N^IM 



2<Ks 



2<Ks 



4 /E l#;l 2 E l/^l 2 </^/Wi + o(/4a X ), 

2<Ks 2<Ks 

where the last two inequalities are obtained by applying Cauchy's inequality and the or- 
thonormality of the /3's. Hence 



By construction 




7T* ( f ) 



PljPlk 



*3 pIiI^i 



p{ + o 




f max 



Finally, by Cauchy's inequality 



KKs 



E 



Pi? 



\ 



Ef E^ 

KKs KKs 



E 

KKi 



Pll 



and the lemma follows in the general case. 

For the case s = n, we take pu = tti and the inequality above becomes an equality. 

I 

Lemma 3. Let M = (X$ , X\ , . . .) be an irreducible and aperiodic, reversible Markov chain 
with transition probability matrix P. We rename the states such that tt\ < tt2 < • • • < 7r n 
is the stationary distribution of M. Let 1 = Ai > A2 > • • • > \ n be the eigenvalues of P. 
Let S be a subset of states and let R be the submatrix of P corresponding to the set of 
indices S X S. Let T(z, S) be the time 

T(i, S) = inf (Xt i S I X 0 = i e S). 



Th 



en 



E(r(z,5))<( , (2+ : )lnn ~ ln7ri + 1 



1 - A r 



1 - Pi 



(1 + 0(1)), 



where A r 



m&X2<i<n | e > 0, and \i\ is the largest eigenvalue of the matrix R. 



Proof: Let the eigenvalues of R be \i\ > ^2 > • • • > fJ- s and let p\ be a positive eigenvector 
of R corresponding to The value T(z, S) is the time when a random walk starting at 
state i G 5 first moves to some state in V — S. The probability that a random walk starting 
at i G S does not leave S within the first t steps is Ejes ^ij '• Tl anc ^ T2 ^ e positive 
values to be determined later. Then 



E(T(., Si) = V V «<» < r, + v V V < 



(>0 k£S 



(n) \ - \ - D (t) 



By Lemma 2, 



E(T(z,S))<r 1 + Wvr. + of 
< n + E + 0 ( 




T2 



■jfc 



k£S t>T 2 




max 

7T ? ; 



< ri + r 2 + + O 

1 — /ii \ 1 — A 




al 



sA 



T-2 



+ 0 



'A Tl u? \ /s 3 / 2 A Tl+T2 

_|_ Q I max Pi J _|_ Q '^ma^ 



Now set 



an 



d 



T2 



7Ti I ~ 111 

1o Sa 

lo gA 



7Ti 1 — A r 



Tl 



n 



max \ n l + e 



where e > 0. With these values 



E(r(z,5))<( , (2+ : )lnn ~ ln7ri + 1 



1 - A r 



1 - /i! 



(l + o(l)). 



We now put together these lemmas to obtain the main theorem for chains that are irre- 
ducible, aperiodic, and reversible . 



Theorem 4. Let M be an irreducible and aperiodic, reversible Markov chain on n states 
with transition probability matrix P. We rename the states such that tt\ < 7T2 < • • • < 7r n 
is the stationary distribution of M. Let 1 = Ai > A2 > • • • > \ n be the eigenvalues of P 
and let A max = max2<Kn |Ai|. Let e > 0 be a constant. Then the cover time C starting 
from any state satisfies 

E(C)<— 1 ((2 + e)n\nn-n\rnr 1 + ( ^) + 

max ^ l<i<jAl<j<i ' ' 

Proof: Let Ti be the first time that i states have been seen. Then C = T n and E(C) = 
Ei<Kn-i E(Ti_|_i — T^. Let Si be any set of states of cardinality i and let fii(Si) be the 
largest eigenvalue of the submatrix of P corresponding to the set of indices Si X Si. Let 
= maxs ; fii(Si). Then via Lemma 3 

E(C)< _ ^ ^2±^^ + _L_y 1 + 0(1]) 



v - -'max 

Ki<n — 1 



< — 1 ("(2 + e )nlnn-nln7r 1+ ^ ( E ^) + 

maX ^ l<z<n H<j<i ' ' 

where the final inequality follows from Lemma 1(b). | 

This theorem easily generalizes to Markov chains that are irreducible but not neces- 
sarily aperiodic as follows. 

Theorem 5. Let M be an irreducible reversible Markov chain on n states with transition 
probability matrix P. Let 1 = Ai > A2 > • • • > \ n be the eigenvalues of P. Let tt = 
(7Ti,7T2, . . . ,7r n ) be the eigenvector corresponding to the eigenvalue 1, normalized so that 
Y2i 7Ti = 1- (If M is aperiodic then tt is the stationary distribution of M.) We rename the 
states such that tt\ < 7T2 < • • • < 7r n . Let e > 0 be a constant. Then the cover time C 
starting from any state satisfies 

E(C)< r ^((2 + e )nlnn-nlnvr 1 + £ ( £ tt,-) Vl + o(l)) . 

2 ^ l<i<n 1<j<i ' 

Proof: Consider the Markov chain M 1 with transition probability matrix P 1 = |(J + P). 
Clearly M' is irreducible, aperiodic, and reversible and E(Ci(M')) = 2E(Ci(M)) from any 
starting state i. Furthermore, if 1 = Ai > A2 > • • • > \ n are the eigenvalues of P, then 
P' has eigenvalues 1 = A' x > A 2 > • • • A' n , where A'- = |(1 + A^). Finally, the stationary 
distribution of M' is 7r. Hence the result follows immediately by applying the previous 
theorem to M' . | 
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Corollary 6. Let M be any symmetric, irreducible Markov chain with transition proba- 
bility matrix P. Then its expected cover time starting from any state satisfies 

where A2 is the second largest eigenvalue of P and e > 0. 

Proof: We apply the previous theorem. In this case TTi = 1/n for all i. | 

We now apply this result to expander graphs. An (n, <f, k)- expander is a (/-regular 
graph G(V, E) on n vertices, such that for every set X C V with \X\ < n/2, its neighbor- 
hood satisfies |T(X) — X\ > k \X\. (For a subset X of V, the neighborhood of X is defined 
as r(X) = {d G F I G £ for some x £ X} .) 

Alon [1] has proven that if G is an (n, <f, &)-expander then the transition probability 
matrix for a random walk on G has second largest eigenvalue A2 < 1 — k 2 / (d(A + 2& 2 )). 

Corollary 7. Consider a random walk on a d-regular expander graph. Its expected cover 
time starting from any state satisfies E(C) = O(nlogn). | 

Corollary 8. Let P be the transition probability matrix corresponding to a random walk 
on a connected graph G. The expected time to visit all states in G satisfies 

„ , ^ n 2 Inn , . . s 
E(C)< T - I - (l + o(l)), 

where A2 is the second largest eigenvalue of P. 

Proof: Let d{ be the degree of node i in G. Then TTi = d{j d{ and therefore TTi > 1/n 2 
for all i. I 

In general the bound of Corollary 8 cannot be improved as the following example 
shows: Let / be the n by n identity matrix and let J be the n by n matrix consisting of 
all l's. Consider the graph G on 2n vertices with adjacency matrix A, where 

A-( J 1 
A ~ \I 0 

This graph has n vertices connected to form a complete subgraph, each connected to 
one other distinct vertex. It is not difficult to see that the cover time of this graph is 
0(n 2 logn). Since A max = 1/y/n, Corollary 8 gives a tight bound for this graph. 
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3. Application to universal sequences. 



Let G be a graph on n vertices. At each vertex v, let the edges incident with v be given 
the distinct labels 1, . . . } d(v) } where d(v) is the degree of v. The labels at the two ends 
of an edge are not necessarily equal, that is, each edge is labeled twice. A sequence a in 
{1, . . . , n}* is said to traverse G from v if, by starting from v and following the sequence of 
edge labels cr, one covers all the vertices of G. (We make the convention that a label not 
present at the current vertex results in a null move.) Let Q be a collection of graphs. We 
say that a is universal for Q if it traverses every graph in (?, from every starting point v. 

The original motivation for obtaining bounds on the lengths of universal sequences 
comes from the attempt to prove bounds on the deterministic space complexity of the 
reachability problem for undirected graphs. Aleliunas et al.[6] have shown that there exist 
sequences of length 0(d 2 n 3 log n) that are universal for all labeled graphs on n vertices with 
degree bounded by d. For (/-regular graphs the bound can be improved to 0(dn 3 log n) 
via the results in [10]. Using the results of Lemmas 1 and 3, we show that there exist 
sequences of length 0(dn 2 log n) that are universal for graphs with A2 bounded away from 
1. 

We start by considering connected graphs that are not bipartite (and hence the Markov 
chain is aperiodic) and then generalize the result to arbitrary connected graphs. 

Lemma 9. Consider a random walk on a d-regular graph G on n vertices. Assume that 
G is not bipartite. Let the corresponding Markov chain have transition prohahility matrix 
P with eigenvalues Ai > • • • > \ n . Let A max = max2< 8 <« A^. Let e > 0. Then the cover 
time C starting from any state satisfies 

Pr ( C> 4nl " n +t]< exp (n ln(t + n) - ^ l ~ h + 0 (l)) 

V 1 - v H /V n J 

Proof: Let v be any starting position. Let the random variable be the first time that 
k vertices are seen, starting from v. Let S be any connected set of states of cardinality k 
and let T(z, S) be the time that a random walk starting at state i £ 5 first leaves S. Fix 
t > 0. Then 

Pr(T k+1 -T k > r + t) < max Pr(T(i,S)>T + t). 

i , S 
\S\=k;i£S 

Let S be the subset of vertices and i the state in S for which the right hand side above is 
maximized, and let R be the submatrix of P corresponding to the indices S X S. Let \i\ 
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be the largest eigenvalue of the matrix R and let /i max = niax2<j< s \fJ-j\- Then following 
the proof of Lemma 3 (applying Lemma 2 and the fact that regular graphs have stationary 
probabilities all equal to 1/n) we derive 

p r (2w, - t> > r + 1) = e = E i?[/ 2 E < + ' * £ ^ E < + ' 

jeS h£S ]£S h£S j£S 



ACQ V 7 ,■«=.<? V^HZ^/ 9 !^ 



fees jes 

Setting t = 41nn/(l - A max ), we get O(Amax) = 0(l/n 2 ) and since /i max < A max and 
/"max < fJ-i, we obtain that 

Pr m+1 _ Tk>T + t)< _ g (I + o (.1)) g ( B& ,. + o (d 

<^(l+o(l))- 
Finally by applying Lemma 1(b), 

kV 



Pr (Tfc+i - T k > r + t) < (l - (1 - A max )^— (1 + o(l)) 

< A ( 1 + o(1) ). 



Let a = (l - (1 - A max )/n). Then 

Pr(T, +1 -T k >r + t) < (^)-^(l + o(l)), 

1 — ax 

where the notation (x f ) f(x) means the coefficient of x f in the formal power series expansion 
of /. Therefore 

Pr (C > r + t) < ( x *) (1 + o(l)) = ( n+ l~ 1 ) (l + °(1)) 

< ( t + n )- a *(i + 0 (i)) 

< exp (n ln(< + n) - t(1 ~^ max) ^) (l + o(l)) . 
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Theorem 10. Let Q he the collection of all d-regular graphs on n vertices with A2 < 1 — c 
for some constant c > 0. There exist universal sequences for Q of length 0(dn 2 logn). 

Proof: First, we generalize the previous lemma as in the proof of Theorem 5, to get rid of 

the requirement that G is not bipartite. We obtain that 

/ 8nlnn \ / , , s t(l - \ 2 )\ , , sN 

Pr ( C> Y^ +t ) ~ 6XP ( nln ( t + n ) ~ 2n + 

Therefore the probability that a randomly chosen sequence o from {0, 1, . . . , <f — 1}* of 
length 8nlnn/(l — A2) + otdn 2 Inn does not traverse any fixed graph G £ Q is bounded by 
exp((2 — acd/2)n\rin + 0(n log log n)) . Since the number of graphs in Q is at most n nd , 
the probability that there exists a G £ Q that <r doesn't traverse is at most exp((2 + d — 
acd/2)nlnn + O(nlog logn)}, which is less than 1 for an appropriately chosen value of a. 
Hence, there exists a sequence a of length 0(dn 2 logn) that is universal for Q . | 

4. The lower bound for rapidly- mixing chains. 

In this section, we will show that a rapidly-mixing doubly-stochastic Markov chain on 
n states, requires O(nlogn) expected steps to visit all states. (A rapidly-mixing chain 
is a chain for which A max < 1 — e, for a constant e.) For example, the Markov chain 
corresponding to a random walk on a regular expander graph satisfies these conditions. 

We use the fact that a rapidly-mixing Markov chain gets close to the stationary 
distribution in time proportional to logn. Since the Markov chain is doubly-stochastic, 
the stationary distribution is uniform. Therefore, if s states have been seen, the expected 
number of new states seen in the next k steps is close to k(n — s)/n, for appropriately 
chosen values of k. This is essentially what we expect if we were to apply a coupon 
collector's argument. By judiciously choosing A; as a function of s and repeating the 
argument, one can prove an O(nlogn) lower bound on the time to see all n states. 

Lemma 11. Let M he a Markov chain on n states with transition prohahility matrix P. 
Assume that P is douhly-stochastic. Let the random variable T s he the first time that s 



states are seen. Suppose t is such that 



pit) 

13 



< -Kr for t > t . Then 



E(T s+r -T s )>\ (^-) - — + 0(1), 
2 \n — s J n — s 

for integers s and r satisfying 0 < s < n, and 0 < r < n — s. 

Proof: Suppose that the random walk has visited a set of states S of cardinality IS"! = s. 
Define Nk to be the number of new states seen in the next k steps and Rk{i) to be 

13 



the number of steps spent in S out of k steps, starting in some state i £ S. Then 
R k (i) = SjGS Xli<m<fc , where X™ is an indicator random variable, which is 1 if the 
random walk has moved from state i to state j after m steps, and 0 otherwise. Therefore, 



1 1 



On the other hand 

x , /.xx n — s , ts (k — t)s 

E(N k ) < k — E(R k (i)) < k + — + y - 

n n n A 

Recall that T r is the time needed to see r different states. Plainly 

E(iV,) 



(4) 



and hence 



Pr(T s+r — T s < k) = Pr(N k > r) < 



E(T s+r -T S ) = J2 Pr(T s +r ~T s >k)>r+ ]T M 



r<k<U 

where U is an upper limit to be determined. Replacing E(N k ) from equation (4), we obtain 



E( 



T s+r -T s )>r+ f 1 



n — s 



k 



TS 



(k - r)i 



r<k<U 



n r nr 



rw 



>L r U(U + 1) fn-s | s 



nr rn- 



U- 



. TS 



nr 



Taking U 



we get 



E(T s+r — T s ) > 



nr 1 / nr 



n — s 2 \ n — s 



TS 



n — s 



+ 0(1). 



Lemma 12. Let M he a Markov chain on n states with transition prohahility matrix P. 
Assume that P is douhly-stochastic. Let the random variable T s he the first time that s 



states are seen. Suppose r is such that 
(a) 



P. 



(t) l 



«J n 



< -\ for t > t . Then 



E(T Ls+rJ - T Ls j)> - ^ 



1 / nr 



nr 



n 



TS 



n — s (n — s) 2 n — s l n — s 
for reals s and r satisfying 0 < s < n, and 0 < r < n — s. 



+ 0(1), 
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(b) 

1 ft 

E(Ti (m+1) „ I - Ti _™ | ) > - — — + 0(rm), 

I m + 2 J l_m + lj Z m + Z 

for any positive integer m. 

Proof: For part (a) we apply the previous lemma and the fact that 

n\r\ n(r — 1) nr n nr nr n 

J > — ^ >- > > 



n — l-s\ n — s + 1 n — s + 1 n — s n — s (n — s) 2 n — s 

For part (b), we substitute in part (a) with s = ™+\ and r = ^ m+1 ^ m+2 ) • We obtain 

E(T| (_rn + l)n I - T| | ) > ]- ( ~ ~ ~ (™ + 1) ) - THl + 0(1) 

L m+2 J lm+i\ z ym + z m + z J 

1 ?7- 

> h 0(rm). 

- 2m+2 V ; 



We now apply Lemma 12 to sets of increasing size to obtain the main result. 

Theorem 13. Let M he an irreducible and aperiodic, reversible Markov chain on n 
states with transition probability matrix P. Assume that P is doubly-stochastic and 
has eigenvalues 1 = Ai > A2 > • • • > \ n . Let A max = max2< 8 <« | Ai |. Suppose that 
Amax < 1 — n e_1 for e > 0. Then the cover time C , starting from any state, satisfies 
E(C) = O(nlogn). 



Proof: From Lemma 2 



P$ = ± + 0(\ t mfa ). 



1 



n 



Since (1 t^tY < -\ for t > "in 1 e lnn, we have 

Applying the previous lemma 



pit) _ 1 

ij n 



< X for t > n 1 ' 6 , S > 0. 



E(C)> J] Effi^.-T, ,)> J] ( l ^— + 0 (rm) 
z — ' I m+2 J Lm+iJ z — ' \zm + z 



0<m<(7 0<m<(7 

> — ^— + 0(n + rU 2 ). 

for any integer £7 > 0 and r > n 1_l5 . 
Choosing U = yn/r, we obtain 

E(C)> nln( 4 n/T) +Q(n). 
Since we can take r = n 1-5 , we obtain E(C) = O(nlogn). 
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5. The lower bound for slowly-mixing chains. 

In order to show that the cover time for slowly-mixing reversible Markov chains (chains 
for which lim n _ i . 00 A2 = 1) is large, it suffices to show that there exists a pair of vertices 
i and j such that the expected first passage time from i to j is large. We obtain such a 
result by proving an identity that relates the weighted average over all pairs of vertices i 
and j of E(ifij), the expected hitting time, to the eigenvalues of the transition probability 
matrix. 

Lemma 14. Let M be an irreducible and aperiodic, reversible Markov chain on n states, 
with transition probability matrix P and stationary distribution (tti , . . . , TT n ). Let Pij(z) = 
S m >o P^iJ 1 ^ z m be the generating function for the probability that a particle in state i moves 
to state j in m steps. Then 

p ij( z ) = S ij + ^3Y~z + A ^{ z \ 

where 

2<k<n V K ' 

and 1 = Ai > A2 > . . . > A n are the eigenvalues of P , and ?/>i, . . . , tp n and (fi, . . . , (p n are a 
biorthonormal basis of left and right eigenvectors corresponding to Ai, . . . , \ n . 

Proof: By the spectral theorem 

p ij( z ) = <fki^kj(^kz) m = + '-Pki^kjTT—^r — v 

m>0KKn l<k<n v K ' 

Since the right eigenvector (fi can be taken as (1,1,..., 1) and the corresponding left 
eigenvector ipi is (7Ti, . . . , 7r n ), we obtain the claimed result. | 

Theorem 15. Let M be an irreducible and aperiodic, reversible Markov chain on n states, 
with transition probability matrix P and stationary distribution (tti, . . . , ir n ). Then 

^ TTiTTj E(Hij) — ^ 



i<i<n 2<k<n 

l<j<n ~ ~ 



where the random variable Hij is the hitting time (or first passage time) from i to j , and 
1 = Ai > A2 > . . . > \ n are tie eigenvalues of P. 

Proof: We start from the well-known identity 



H tJ (z) = ^ Pr(H tJ = m)z 

m>0 
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Because M is reversible 



TTiPij(z) = irjPji(z). 
Therefore, via the previous lemma, we obtain that 



E wi H n( z ) = E p-±) E ^;(*) = E E 

i<i<n KiXn JJ ^ ' Ki<n KiXn JJ ^ ' Ki< 

l<j<n - - - - 

v _A l _ = v — 

4r^. P^(z)(l-z) ^ (1 



Differentiating with respect to z and setting z <— 1, we obtain 

E ^'E(i^)= E (l-T i + A ii (l))=n-l+ E 



i<i<" KiXn 2<k<n ^ k 

l<j<n — — — — 



Corollary 16. Let M be an irreducible and aperiodic, reversible Markov chain on n states, 
with transition probability matrix P. Let 1 = Ai > A2 > • • • > \ n be the eigenvalues of P. 
If A2 > 1 — n i nn then the expected cover time starting from the stationary distribution of 
M is 0(n log n). 

Proof. If A2 ^ 1 ulnn' then Theorem 15 implies that 

l<i<n El<j<n Ki^j E (^j) ~ 

O(nlogn). Hence, there is a j such that Yli< i< n 77 i = ^(^logn). But the expected 

cover time starting from the stationary distribution is defined as 

E(CV)= E ^ E ( C *)> E ^E(^ J ) = 0(nlogn). 

Ki<n Ki<n 



The following corollary was also proven by Landau and Odlyzko [12] via a completely 
different approach. 

Corollary 17. Let P be the transition probability matrix of an irreducible and aperiodic, 
reversible Markov chain on n states, corresponding to a random walk on an undirected, 
connected graph G, with maximum degree d. Let 1 = Ai > A2 > A2 > • • • > \ n be the 
eigenvalues of P. Then 

A 2 < 1 1 



n 2 d 
17 



Proof: Aleliunas et al.[6] show that for any undirected, connected graph with maximum 
degree d, E(Ci) < dn 2 , for all i. Thus we have 

\^ — = \^ 7Ti 7Tj E(Hij) < maxE(Hij) < maxE(Ci) < dn 2 . 

^ — ' 1 — At- ^ — ' i,?' i 

2<k<n i<i<n 

~ ~ l<j<n 

Clearly 1/(1 — A&) > 0, for k > 2. Hence, 1/(1 — A2) < c/n 2 , and the corollary follows. | 

Mazo [14] considered a closely related measure to our average first passage time, 
denoted iV, and defined by 

N = -A r V V E(#„). 

v ' l<i<n i<j<n 

Mazo was interested in determining which chains minimize and maximize the quantity N 
and showed via a rather difficult proof that for random walks on connected, undirected 
graphs, N is minimized for the complete graph. He also conjectured that it is maximized 
for the bar-bell graph. Using our formula, we prove both results for the quantity N' = 

^l<i<n TViTVj~E(Hij). 

1 < j < n 

Corollary 18. Consider tl tl ¥3>H dom walk on an undirected graph with n vertices and no 
self loops. Let 

N'= ]T ^E(i^). 

1 < i < n 
l<j < n 

Then N' is minimized for the complete graph. 
Proof: We have shown that 

N '= T. rrv < 5 > 

2<k<n 

Because the trace of the transition probability matrix is 0, and the trace is equal to the 
sum of the eigenvalues, we have ^2 2 <k<n ^ k = ~~ 1- But ^he function on the right-hand side 
of (5) is Schur-convex (see [13] pp. 54-58) and hence is minimized when A& = — ^-j- for all 
k, 2 < k < n. Since these are precisely the eigenvalues of the Markov chain corresponding 
to the random walk on the complete graph, the claim is proven. | 

Corollary 19. Consider a random walk on an undirected graph with n vertices. Let 

N'= ^vr.E^). 

1 < i < n 
l<j < n 



Then N' 



0(n 



) for all graphs and N' = 0(n 3 ) for the bar-bell graph. 



Proof: Again we use the bound E(Ci) < n 3 for all i. Thus we have 



1 



7Ti 7Tj E(Hij) < maxE(r7ij) < maxE(Ci) < n 



3 



2<A;<n 



1- A fc 



1 < i < n 
l<j<n 



and so N' = 0(n 3 ) for all graphs. But for the bar-bell graph Landau and Odlyzko [12] 
have shown that A2 = 1 — (for some constant c), and the corollary follows. | 
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